Autostereoscopic displays are stereoscopic displays that do not require eyewear selection devices. Such displays have been the subject of inventive scrutiny for almost a century. An important step was taken in 1908 with Lippmann's invention of integral or fly's-eye photography that resembles holography in its effect. Since then, numerous inventors have sought to create a binocular stereoscopic illusion using an optical selection technique located at the display surface (photographic print, motion picture screen, electronic display).
The raster barrier is one approach that has benefits in terms of simplicity of fabrication and optical quality, but it suffers from low brightness. Another approach, attributed to Ives, is an offshoot of Lippmann's work, and uses a series of elongated lenslets or lenticules as a selection device. While a lenticular screen is more difficult to fabricate than a raster barrier screen, it has greater brightness. The lenticular screen has found commercial acceptance and there is substantial literature regarding this art, with interest accelerating judging by recent patent activity.
Since stereoscopic depth information is represented by horizontal parallax, Ives designed a lenticular screen, corduroy-like in structure, with lenticules having only a horizontal focal length. Unlike Lippmann's fly's-eye device, Ives' device restrained the stereoscopic image information to within vertical going sections which are placed in juxtaposition with a corresponding lenticule. Ives called his device the parallax panoramagram (it's also known as the lenticular stereogram) because, in addition to the stereoscopic effect, since multiple views are provided, the resulting display has a look-around capability (if only in the horizontal direction). The term view in this disclosure refers to a single one of the multiple individual perspective images that make up the content of the lenticular display.
Although this look-around capability is interesting, the value of having multiple views is that the result is an acceptable viewing zone. The look-around capability is of secondary importance, since the observer's mobility is usually limited. Look-around capability is an attribute of the multiple perspective nature of the display, without which there could not be an extended viewing zone.
There have been many attempts to perfect the parallax panoramagram. The technique has taken on various embodiments and has been used for commercial portraiture, amateur photography, and in mass-produced printing for advertising, novelties, magazine and book illustrations.
The art of producing parallax panoramagrams has progressed because it provides advantages such as a good stereoscopic effect, adequate viewing angle, full color, good sharpness (at least for the principal area of interest, usually having low parallax values), and decent mass manufacturing techniques have been developed. Nevertheless, an examination of panoramagram images produced fifty years ago and those produced today shows no improvement in stereoscopic effect and image quality.
Although the result resembles that produced by a stereoscopic hologram, the panoramagram does not require laser light for photography, has no speckle noise, and has superb color reproduction. In almost every way, panoramagram images are superior to hologram images. The single way in which holograms excel is that they can have a greater viewing zone.
One improvement to the art forms a panoramagram by combining individual images created in a perspective sequence by a multiple camera array. This technique directly enables the current efforts to create lenticular stereograms by means of computer technology. Prior to this improvement, panoramagrams were photographed with a moving camera having a moving shutter made up of Ronchi-grating-like slits, or with a camera and a horizontal going slit-aperture and lenticular screen at the film plane. These and other approaches shared one common approach: they turned temporal parallax into spatial parallax. A disadvantage of this technique is that it requires a subject that does not move since the exposure takes place over a period many seconds. The multiple camera approach, on the other hand, allows for a series of exposures of the required perspectives taken at the same instant so that moving subjects may be photographed. Hence the multiple views required for a panoramagram are produced as individual images which are combined to simulate the panoramagram effect.
The advantage of the original panoramagram, in which parallax is captured temporally, is that it provides a kind of continuum of views from the leftmost to the rightmost perspective. Upon viewing the panoramagram, when moving one's head to the left or to the right, the result is a continuously varying perspective change like that which occurs in the visual world. The term a kind of continuum is used because, in fact, the image changes from left to right are discrete since the image recording medium, namely silver-based photographic materials, have a finite resolution resulting from well understood optical and photographic limitations.
Workers in the field realized that a panoramagram can be synthesized from interdigitated or vertically sliced and assembled component elements. They accomplished this slicing and rejuxtapostioning of image sections optically. This process is called interdigitation in this disclosure, but elsewhere it is sometimes referred to as interleaving.
FIG. 1 shows the gist of the procedure, in an abbreviated form, since the technique is thoroughly covered in the prior art literature. The purpose of this explanation is to give some background and context to the invention disclosed without necessarily serving as a tutorial.
Subject 101 is photographed by a camera array made up of cameras 102, 103, and 104 (cameras 1 through 3). For purposes of illustration, the number of cameras and images is restricted to three, but as the reader will understand, the number is arbitrary and used in order to simplify the explanation.
Photographs 1, 2, and 3 (105, 106 and 107) of subject 101 are produced by respective cameras 102, 103 and 104. These perspective views are called master images. In some contexts, the master image may be referred to as a component image to emphasize that it is being operated on as one of a series of images. The arrows 117 connecting the cameras with their respective master images indicate the photographic process. The master images are sliced vertically into multiple segments and then aligned next to each other as indicated by arrows 112. The result 113, is the print component of the interdigitated stereogram that replicates the characteristics of the panoramagram. We refer to bracketed sections 108, 109 and 110 as columns, and each column contains, in this case, three stripes, 1, 2, and 3. The stripes are signified as 3a, 2a, 1a, and then 3b, 2b, 1b, and so on, to indicate that they come from different portions of the images 105, 106, and 107.
The stripes are inverted with the left perspective view at the right end of the column and the right perspective view at the left end of the column because of the optical inversion caused by the individual lenticules 114, 115, and 116 of lenticular screen 111. Lenticular screen 116 is made up of refractive elements which have a horizontal focal length only, and these elements or lenticules 114, 115, and 116, allow the left and right eyes to see the perspective views required for a stereoscopic image. Each lenticule is juxtaposed with an associated column and its stripes. Thus, lenticule 114 is associated with column 108 and its stripes 3a, 2a and 1a; lenticule 115 is associated with column 109 and its stripes 3b, 2b and 1b; and lenticule 116 is associated with column 110 and its stripes 3c, 2c and 1c. 
It is easy to imagine the master images (if they are paper prints) 105,106, and 107 being sliced up with a scissors and then laid next to each other side-by-side and pasted together with this sequence repeating for other segments of the photographs in the sequence of columns described above. If indeed this were the method employed to create the interdigitated stereogram, and none of the slices was discarded, the resultant interdigitated image would be three times as wide as the original master images 105, 106 and 107, receptively. The resultant image would be anamorphically stretched and would look peculiar. Therefore, only one out of every three strips cut up from images 1, 2, and 3 needs to survive to result in an undistorted image. The purpose of this scissors and paste explanation is expository, but we will return to this example and the principles inherent in it.
Until recently, workers in the field interdigitated master images optically. With the widespread use of the computer, there has been a substantial extension of the art because the computer can be made to emulate scissors and paste or optical techniques and can easily re-scale images anamorphically. The image may be re-scaled in either the vertical or the horizontal direction to result in images that are not anamorphically stretched, or, stripes (or pixels) might be thrown out to obtain the same result. The beautiful thing about computer techniques is that they can be used in algorithms that break up a large and complex series of operations into smaller manageable units of computation. The algorithms are then able to operate on a pixel by pixel level to reshuffle, average, or rearrange pixels to conform to the topological needs of our specific problem.
It has been assumed in the above description that we are using the classical and almost universally employed Ives' technique in which the lenticules are oriented so that their boundary edges are parallel to the vertical edge of the display or print. In 1968, Winnek made a tremendously important improvement as disclosed in U.S. Pat. No. 3,409,351. In his variation, the lenticules and associated columns and stripes are laid down at a diagonal or at some angle relative to the edge of the print or display.
The purpose of the Winnek improvement is to increase the horizontal resolution of the display and to reduce or eliminate moire patterns resulting from the optical interaction of lenticular screens. Winnek was concerned with means for copying and resizing (enlarging) lenticular stereogram prints. Winnek's idea is used by Street, as disclosed in U.S. Pat. Nos. 4,668,063 and 4,757,350. Sandor et al. applies the Winnek concept to raster barrier displays as disclosed in U.S. Pat. No. 5,519,794. Sandor is concerned with increasing the horizontal resolution of the display.
From Winnek, we learn that “the present invention is designed to afford means . . . whereby moiré and other unwanted patterns are avoided in the ultimate print.” Winnek also discusses the improvement in sharpness made possible by the technique. Sandor is explicit about the benefits, stating that “This rotation results in increased numbers of images per unit pitch, by trading vertical for horizontal resolution.”
Given Winnek's disclosure, it is a short step to apply the idea to flat panel displays, as was done by van Berkel in European Patent Application 97200399.0. Van Berkel uses a perfect fit approach since his combination of pixels and lenticules must be juxtaposed with high precision in order for the process to work. As the reader shall see, our techniques use the more forgiving approaches of best fit and pixel averaging.
Interestingly, unlike the art described by van Berkel, which requires slanted Winnek lenticules, our approach works with slanted or non-slanted lenticules. For paper prints using common desktop printers, the resolution is so high there may well be little or no advantage to using the Winnek technique. However, for flat panel displays in which the pixel density is much lower, the Winnek technique definitely works best to reduce the effect of distracting moire patterns and to increase the horizontal resolution.
One point of comparison with van Berkel reveals that our techniques work with slanted lenticule systems in which there are repeating groups of display pixels, though our techniques can also work with systems where that is not the case. Our techniques apply regardless of the angle of lenticular slant, even if the amount of lenticular slant is zero degrees, as mentioned above. Our techniques apply regardless of the pitch of the lenticular elements.
Another comparison with van Berkel indicates that our techniques do not address the shape or other characteristics of the lenticular element. However, we do assume that the optical properties of the lenticular element are such that, if a range of views appears underneath a lenticular screen with right-eye views towards the left edge of each lenticule and left-eye views towards the right edge of each lenticule, distributed more or less linearly, then the image will appear stereoscopic.
Comparing further with van Berkel reveals that our techniques apply whether the interdigitated “pixels” are whole RGB (red, green, blue) pixels or single-color sub-pixels. Our techniques are not particular to any given style of color sub-pixel layout, or a particular sub-pixel layout. If the location of a pixel or sub-pixel can be calculated with respect to the placement of the lenticular elements, then our techniques apply.
FIGS. 4a and 4b illustrate Winnek's system. In FIG. 4a, we see interdigitated print or display surface 404 made up of columns 401, 403, 403 with each column having its associated series of stripes 3a, 2a, and so on, as described above with respect to FIG. 1. Each column is in juxtaposition with its appropriate lenticule 406, 407 and 408 of lenticular screen 405. Unlike the arrangement shown in FIG. 1 in which the boundary edge of the lenticular columns is parallel to the vertical edge of the display, the lenticules, or more properly the lenticular boundaries, are tipped at some angle Φ to the vertical going edge of the display. The columns, stripes, and lenticules are all tipped at an angle Φ, formed by the running edge of the column, stripe, or lenticule, with the vertical edge of the print or display. For a Winnek system display, Φ is greater than zero but less than ninety degrees.
FIG. 4b shows the overall view and possibly more clearly illustrates the geometry. Display or print 409 is made up of columns 411, which are parallel to the lenticules and which are tipped to the vertical edge 412 of the display 409 by angle Φ, which is preferably between five and ten degrees.
Interest in the medium has waxed and waned over the years. In the past ten years or so, many patents have been issued in the field, most probably because of three developments: The personal computer, desktop printers, and flat panel displays. Not only do these inventions create additional possible applications, but they also allow inventors to vary image parameters and view the results rapidly. Moreover, these technologies allow for new approaches to interdigitate image elements, which have advantages over the optical equivalent. For one thing, the technique lends itself to computer generated image applications with a three-dimensional database.
Prints made on desktop printers and images produced on flat panel displays share an interesting characteristic: their pixels are specified and located by means of a Cartesian grid and they have a flat surface. Cathode ray tube displays, on the other hand, are usually not flat and the location of each pixel with exact specificity cannot be guaranteed. However, modern printers and flat panels overcome these objections because of the intrinsic nature of these display devices.
The present disclosure, in part, presents a means for reducing the amount of information required in the master images. An interesting example of prior art is U.S. Pat. No. 6,023,263 to Wood. The essence of this patent is that, rather than rendering N different views of a scene, all at high quality, Wood would render 2 (or perhaps more than 2 but less than N) views of the scene. Wood would then obtain the other scene views by deriving one scene view from a different scene view's actual rendering.
Thus, in a 4-view example, views 2 and 3 would be rendered normally, and then views 1 and 4 would be calculated, not by doing two additional renderings, but by applying displacements to regions of views 2 and 3. This would require some integration of Wood's technique with the actual rendering routines utilized. Conversely, the technique of the instant invention would generate all 4-view (generally speaking N-view) renderings using standard rendering methods, except at a lower resolution than the final overall resolution.
The advantage of the Wood technique over that which is disclosed here is that it has the potential to reduce rendering overhead considerably. The advantage of our technique is that it is more versatile, particularly when used with standard rendering software. In point of fact, in such a useful and important case, applying the Wood art would be inconvenient or impossible.
In the most general sense, the present invention describes creating an autostereoscopic display using a lenticular screen as a selection device. The lenticules may follow the traditional vertical going arrangement, in which the boundary edge of the lenticules is parallel to one edge of the display, first enunciated by Ives, or the improvement using a slanted or tipped angle for the lenticules with respect to one edge of the display. Generally, for flat panels and their sparse pixel density, Winnek's approach has advantages. In addition, these displays tend to have black interstices between the pixels, the visibility of which tend to be exacerbated by vertical going lenticules because of their horizontal magnification. Happily, tipping the lenticules, as described by Winnek, reduces the visibility of the interstices.
The invention consists of two related components that together form the basis for a practical autostereoscopic display with excellent pictorial quality. The first portion of the invention specifies means for minimizing the information content of master images with distinct perspective views, and the second concerns itself with re-mapping the pixels of those master images to conform to the needs of the selection device.
Because the master images are created according to our prescription for minimum content while creating an excellent pictorial result, the rendering time for a graphics application is not increased compared to a conventional planar rendering. This is an important consideration in many applications in which no sacrifice in smoothness of animation or interactivity can be tolerated by demanding users.